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Abstract 

In the last fifty years, the use of stories in education has included vignettes as an effective stimulus for discussion 
of real-life contexts and problems. Hozvever, vignettes have rarely been used as an assessment tool and there is no 
reported consensus on their definition and design. This article documents the use of vignettes as an effective method 
of assessing pedagogical understanding in our teacher development courses from 1995-2003, suggesting that vignettes 
are significantly correlated with more traditional forms of assessment, are highly predictive of course-ending project 
performances, and represent an episode of learning in their own right. Finally, we propose a more concise definition 
and a more rigorous course of study for vignette development and implementation. 


A critical theme of institutions of higher 
education in the preparation and continuing 
education of teachers is that educational research 
informs and is informed by practice. Identifying 
and assessing what and how teachers learn during 
their professional development is an important 
component in assisting them to better teach their 
students. The use of stories in various formats has 
long been a powerful and successful method for 
modeling, teaching, and researching behavior and 
understanding in general education, health sci- 
ences, social sciences, and behavioral sciences. 

One type of story, the vignette, has been used 
in the social sciences and more recently in the 
health and education fields as a reflective, re- 
search, and modeling tool (Callicott, 2003; Camp- 
bell, n.d.; Chambers, 1999; Galguera, 1998; Hughes 
& Huby, 2002; King, Murray, Salomon, & Tandon, 
2002a; Kruse, 1997; New Hampshire Equity Hand- 
book Writing Team, n.d.; Pransky & Bailey, 2002; 
Schwartz & Riedesel, 1994; TESOL, 2001; Vaughn 
& Klingner, 1999; Volkmann, 1998). Researchers 
and educators have found vignettes to be very 
effective in these contexts for several reasons: 
vignettes are relatively easy to construct, they 
provide a useful focus and stimulus for discus- 
sion, they are valuable in addressing difficult-to- 
explore and sensitive topics, they can be used with 
individuals and groups, and they reflect real-life 


contexts and problems. 

Vignettes have rarely been reported as a 
means of assessing mastery of course content. 
Furthermore, there are few definitions or con- 
struction criteria for vignettes in these contexts, 
and certainly no reported consensus on their 
definition and design. Given the benefits of using 
vignettes in research, modeling, discussion, and 
reflection, and the need to determine teacher 
qualification, it is important to determine whether 
vignettes can also be an effective assessment tool 
in teacher education. 

An important step in this process is the collec- 
tion of evidence that vignettes are a reliable as- 
sessment tool that complements more traditional 
assessment styles. In addition, it should be deter- 
mined whether vignettes can augment the infor- 
mation gathered in more traditional assessments. 
Besides content and skill mastery, do vignettes 
help to measure affective goals, such as the moti- 
vation to learn? Do vignette assessments provide 
evidence of problem solving and critical thinking? 

Furthermore, it is critical to determine if 
vignettes can assist in state-mandated assessment 
of teacher understanding of pedagogy. The chal- 
lenge of assessing portfolios may be partially 
addressed through vignette scoring guides that 
quantify responses that are unique and authentic. 
Finally, vignettes should be studied for evidence 
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of an assessment as an episode of learning. Does 
an assessment task that encourages modeling, 
rehearsal, and transfer provide more opportuni- 
ties for learning than more traditional forms of 
assessment, (e.g., multiple-choice, fill-in items)? 

Since 1995, we have been using vignettes as a 
method of assessing pedagogical understanding 
in teacher development courses. We believe the 
vignette to be a highly effective assessment tool 
and suspect that vignettes have a strong predic- 
tive component as well as provide a rich learning 
experience for eliciting and promoting teacher 
understanding. We would like to share our experi- 
ences and data with teacher educators and re- 
searchers in order to propose a course of study on 
the use of vignettes in assessing pedagogical 
understanding in teacher education programs. 
Based on the literature and our experiences, we 
would also like to propose a more concise defini- 
tion and establish more rigorous criteria for 
vignette development and implementation in this 
assessment process. 

Review of Literature 

Stories as Educational Tools 

A story is defined by Cambridge Advanced 
Learner's Dictionary (Cambridge University Press, 
2003) as "a description, either true or imagined, of 
a connected series of events and, often, the charac- 
ters involved in them." Stories have been used in 
many different forms and formats as a powerful 
and successful method for modeling, teaching, 
and researching behavior and understanding in 
general education, health sciences, social sciences, 
and behavioral sciences. Story formats include case 
studies, long narrative descriptions of real or 
hypothetical situations in which learners are 
asked to identify or solve a problem (Gideonse, 
1999; Hrabe, Kinzie, & Julian, 2001; Jackson, 1998; 
Loughner, Harvey, & Mil-heim, 2001; Marsick, 
1998); case stories, stories that simulate the real 
world but are written by individuals within the 
classroom and told from their perspectives (Hunt- 
er & Hatton, 1998; Maslin-Ostrowski & Ackerman, 
1998); scenarios, narrative descriptions that contain 
a set of realistic assumptions and facts about the 
future used to provide a unified context for 
decision-making (International Centre for Devel- 


opment Oriented Research in Agriculture, n.d.; 
Pesonen, n.d.), and vignettes. 

Vignettes in Education and Research 

A recent review of the literature on vignettes 
revealed a wide range of definitions, usage, con- 
struction, testing, and examples. Definitions were 
rarely provided and varied greatly, which caused 
confusion of vignettes with other types of stories. 
In fact, many studies and articles used the terms 
case study, case story, scenario, and vignette 
interchangeably, incorrectly, or uniquely, (e.g., 
Jackson, 1998; Kruse, 1997; Walen & Hirstein, 
1995; Weigle & Scotti, 2000; Weiner, Rand, Paga- 
no. Obi, Hall, & Bloom, 2001). 

Broadly defined, vignettes are "short stories 
about hypothetical characters in specific circum- 
stances to whose situation the interviewee is 
invited to respond... moving from the abstract to 
context-specific" (Finch, 1987, p. 106). Vignettes 
consist of text, images, or other forms of stimuli, 
ranging from short written prompts to live events, 
to which research participants are asked to re- 
spond (Hughes & Huby, 2002). Herman (1998) 
uses the term case vignette to describe a written 
description, photograph, or videotaped scene as a 
brief glimpse of an educational situation. Camp- 
bell (n.d.) comes closest to an operational defini- 
tion: 

A vignette is a short story without an ending. 
It is short, but not too short to present an 
issue. It is detailed, but not so detailed that the 
underlying issue gets lost. A vignette presents 
an issue, such as the under-representation of 
girls in advanced math courses, in a context 
with which individuals can identify. A good 
vignette has fewer complexities and personali- 
ties than real life, sets up a situation in which 
there is no one "right" answer, and is flexible 
enough that individuals from different groups 
(teacher/ administrator, female/ male, liberal/ 
conservative) can identify with the story and 
bring their perspective forward in discussions 
of solutions, (p. 2) 

Vignette Construction and Design 

In addition to a lack of consensus on vignette 
definition, there was a noticeable lack of informa- 
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tion on how vignettes were constructed or should 
be constructed. When using vignettes to promote 
discussion and problem solving concerning equity 
in math and science education, Campbell (n.d.) 
proposed a vignette construction process and 
shared samples of vignettes for different target 
groups, (e.g., administrators, teachers, students, 
parents, policy makers, researchers). She listed 
three major steps in creating vignettes: determine 
issues and areas of concern, develop situations 
that are realistic and relevant, and test the vi- 
gnettes on groups similar to those who will be 
using them (Campbell, n.d.). 

In their review of the use of vignettes as a 
research tool in the behavioral sciences and health 
care, Richman and Mercer (2002) proposed an 
alignment along eight axes in the construction and 
application of vignettes. These axes addressed the 
function of the vignette, method of implementa- 
tion, collection, type of respondents, content 
covered, time available, type of response elicited, 
and ethical issues. Richman and Mercer put forth 
the design issue of whether to use already existing 
data in the construction of the vignette or to 
construct material suited to the specific purpose. 
They also posed the question of vignette delivery 
media — written documents, audiovisual, or oral — 
while stating that the literature strongly suggested 
that vignettes are usually given in written docu- 
ments of varying lengths. 

In their review of the application of vignettes 
in social and nursing research, Hughes and Huby 
(2002) addressed the differences between vig- 
nettes and real life processes and explored some 
practical advantages and pitfalls of using vi- 
gnettes. They found that vignettes provided a 
useful focus and stimulus for discussion, may be 
constructed from unrealistic events and real life 
events, were valuable in detecting subtleties and 
nuances, and were useful in addressing difficult- 
to-explore and sensitive topics. The issues they 
presented in constructing effective vignettes 
included making the topics and contexts relevant 
to the audience and addressing the readers abili- 
ties and styles. 

King, Murray, Salomon, and Tandon (2002b) 
addressed similar issues when constructing 
vignettes for surveys: how and when to address 


the vignette character's intellectual, cultural, and 
religious background as well as physical attributes 
and personal information. They recommended 
that vignettes be written so that people from 
different backgrounds could understand them as 
similarly as possible, that details be scrutinized to 
avoid introducing unwarranted assumptions, and 
that vignette characters be as similar as possible to 
the audience. 

When authors did provide samples of their 
vignettes, a wide range of formats were found that 
included short and long dialogues, formulas, 
pictures, and short and long narratives with and 
without follow-up questions. Vignettes ranged 
from 25 to 1000 words in length. The accompany- 
ing questions and tasks varied in terms of num- 
ber, length, type, and amount of detail. 

Vignette Usage 

The lack of consensus of vignette definition 
and format may be due to its diverse usage across 
fields that seldom overlap or confer with one 
another. Vignettes have been used in the social 
sciences since the 1950's (Hughes & Huby, 2002) 
and more recently in the health sciences to model 
best practices (Callicott, 2003; Orlander, Gupta, 
Fincke, Manning, & Hershman, 2000), to describe 
common situations and conditions (Pickett, 
Streight, Simpson, & Brison, 2003; Stelmachers & 
Sherman, 1990), to study technique effectiveness 
(Goldie, Schwartz, McConnachie, & Morrison, 
2001), and to identify attitudes and beliefs (Sleed, 
Durrheim, Kriel, Solomon, & Baxter, 2002; Thur- 
ber, Heller, & Hinshaw, 2002). 

Vignettes have also been used in the field of 
education as models of effective teaching (TESOL, 
2001; Vaughn & Klingner, 1999), to identify and 
study attitudes and beliefs (Galguera, 1998; 
Schwartz & Riedesel, 1994), and as a tool to sup- 
port teachers in their development, reflection, and 
problem-solving abilities (Campbell, n.d.; Kruse, 
1997; New Hampshire Equity Handbook Writing 
Team, n.d.; Pransky & Bailey, 2002; Volkmann, 
1998). Only a few instances of vignettes used to 
assess knowledge were found, (e.g., Cohen, Shete, 
Seal, Daum, & Lauderdale, 2003), and those were 
in the health field. 
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Vignettes as a Written Assessment Tool in Teacher 
Development 

Richman and Mercer's (2002) view of vignettes 
as a feasible alternative to observation and a 
flexible and fertile component of qualitative 
research suggests that a vignette analysis assign- 
ment might be an assessment tool to complement 
more traditional forms of content or skill mastery. 
Research conducted in the fields of health and 
behavioral science (Hughes & Huby, 2002) sug- 
gests that vignettes could be used as an assess- 
ment tool to collect evidence of mastery of skills, 
terms, and concepts for the following reasons: 

1. Vignettes provide a useful focus and stimulus 
for discussion. 

2. Vignettes may be constructed from unrealistic 
events and real life events. 

3. Vignettes are valuable in detecting subtleties 
and nuances. 

4. Vignettes are useful in addressing difficult-to- 
explore and sensitive topics. 

5. Vignettes can quickly generate considerable 
amounts of data from a large participant 
group. 

6. Vignettes can be defined and standardized to 
enable all participants to respond to the same 
stimulus. 

7. Vignettes do not necessarily require partici- 
pants to have in-depth knowledge of the 
topics under study. 

Given the positive aspects of vignette use, we 
performed an analysis of data collected between 
1998-2003 in two online teacher education courses 
that used vignettes as an assessment task to elicit 
teacher pedagogical understanding. Based on the 
research literature and our experiences and re- 
sults, we believe that vignettes are a strong candi- 
date for assessing teacher understanding and 
predicting appropriate teacher implementation of 
instructional strategies. 


Method 

Vignette Definition and Construction Criteria 

We define vignettes as incomplete short stories 
that are written to reflect, in a less complex way, real- 
life situations in order to encourage discussions and 
potential solutions to problems where multiple solu- 
tions are possible. 

The five criteria for this type of vignette are: 

(1) It is a story. It is a narrative but not a dia- 
logue, case study, case story, or scenario. 

(2) It is short. Its length is 50-200 words. 

(3) It is relevant. It simplifies a real-life situa- 
tion that is relevant to participants but one 
in which no participant is likely to have 
expertise. 

(4) It alloivsfor multiple solutions/answers and is 
intended to encourage independent thinking 
and unique responses. It includes a prompt 
with instructions and a set of tasks, i.e., 
specific issues to be addressed in the partic- 
ipant's response directly connected to a 
scoring guide. 

The use of vignettes creates an assessment 
situation where authenticity of authorship is less 
likely to be compromised. Unlike forced-choice 
assessment, (e.g., true/false, multiple-choice, fill- 
in), vignette responses are open-ended and allow 
for more unique demonstrations of content mas- 
tery. Each participant chooses and defends a 
position during the analysis of the vignette. The 
evaluator uses a scoring guide to verify content 
mastery and determine if the work has been 
plagiarized from a classmate or another source. 

(5) It is purposely incomplete. It can be truncated 
— plot line stops at a critical juncture and 
participants complete the vignette — or 
abridged — story's details are omitted so 
that multiple interpretations can be de- 
fended. 

In a truncated vignette participants are asked 
to complete a storyline according to a set of crite- 
ria defined by the course curriculum. Truncated 
vignettes are typically used to evaluate process, 
rather than product, i.e., examining a student's 
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completion of the storyline to assess if a particular 
problem-solving skill has been mastered. In an 
abridged vignette participants are asked to dem- 
onstrate mastery of course content by answering 
specific questions concerning the vignette and 
justifying their positions. Abridged vignettes are 
typically used to evaluate product, rather than 
process, i.e., examining a student's analysis of a 
situation to assess if specific knowledge has been 
mastered. 

The following truncated vignette concerning 
classroom management is drawn from a teacher 
development course taught by one of the research- 
ers: 

A colleague of yours, Bernie Bunsen, is a 
middle school science teacher who designates 
the last 30 minutes of each class meeting for 
one week to complete lab activities on the 
various states of water. To the delight of his 
classmates, one student has been fooling 
around during the lab time, tossing ice cubes 
around the room, sabotaging the fog machine, 
disrupting other group work, etc. Some of his 
classmates are starting to imitate him to get 
the same reactions from others. Mr. Bunsen is 
obviously concerned about the lack of produc- 
tive work getting done as well as the safety 
hazards caused by the misbehavior. Mr. Bun- 
sen has confided in you that he is deciding 
among several options: send the lead misbe- 
having student out of the classroom for deten- 
tion during lab time, assign him more science 
work, fail him on the lab assignment, punish 
the entire class for the misbehavior, speak 
privately with the student, speak with the 
entire class about the misbehavior and the 
importance of safety measures, give the stu- 
dent the option of re-submitting his lab work 
for a partial score increase. 

Complete the following tasks: 

(1) Using the information provided in this 
week's readings, identify the type of mis- 
behavior or mistaken behavior the disrup- 
tive student in the vignette is exhibiting 
and discuss possible causes for his behav- 


ior. Be sure to identify clues in the vig- 
nette, information from the readings, or 
assumptions you made to defend your 
response. 

(2) Choose from Mr. Bunsen's options, the 
readings' list of classroom management 
strategies, and/or your own ideas to rec- 
ommend a set of options for Mr. Bunsen. 
Be sure to defend your selections. 

Note the range of possible responses to these 
tasks. Although the classroom management 
problem is clearly identified in the vignette, the 
cause is not and there are numerous ways for Mr. 
Bunsen to solve the problem, e.g., clarify the 
assignment, remind students of consequences, be 
consistent and caring with the students, and help 
the misbehaving student change his behavior. 

The truncated vignette task is more open- 
ended than that of the abridged vignette, which 
provides scaffolding for multiple interpretations 
linked to the course content. The following abridg- 
ed vignette concerning scaffolding is drawn from 
the same teacher development course: 

Random Guess is a high school math teacher 
who provides a free, two-hour problem-solv- 
ing workshop for juniors and seniors prepar- 
ing for their college admissions exams. Once a 
week after school during the fall semester, 
students meet with Mr. Guess to work on 
sample math problems. This year, he has 
decided to try out some new techniques in his 
workshop. In addition to simply answering 
questions posed by the students concerning 
problems in a handout that he has given them 
the prior week, he has decided to (a) review a 
math topic; (b) solve one or two related prob- 
lems on the board while thinking out loud; (c) 
work with the class to solve several more 
problems on the board; (d) administer a short 
individual quiz of selected problems; and (e) 
review the quiz problems before handing out 
next week's handout of problems. 

Complete the following tasks: 

(1) Analyze what Mr. Guess is doing to help these 


Volume XXVII • Number 1 & 2 • Fall 2004 & Spring 2005 21 




Carolyn Jeffries & Dale W. Maeder 


students learn in terms of Vygotsky's theories 
and their instructional implications in relation 
to scaffolding and social learning. Be sure to 
defend your answer with examples, clues in 
the vignette, or assumptions that you have 
made. 

(2) Describe two ways Mr. Guess could modify his 
instructional plan to promote input from 
others. 

Note again the range of possible responses to 
these tasks. There are several clues within the 
vignette to indicate evidence of scaffolding in Mr. 
Guess' teaching and there are numerous ways to 
promote student input, e.g., cooperative learning 
groups, student demonstrations, teach-back 
activities. 

Vignettes differ from case studies which are 
longer, more detailed reports to be analyzed and 
discussed. Nor is the vignette a case story which 
is written by the student-participant and focuses 
on his/her perspective. Finally, vignettes are 
distinguished from scenarios which are future- 
based stories. 

Setting and Context 

Evidence supporting the use of vignettes as an 
effective learning tool and means of assessing 
participants' pedagogical understanding was 
collected in over 30 sections of two online teacher 
education courses between 1998 and 2003. The 
participant population consisted of employed 
school teachers, college instructors, and corporate 
trainers. The lectures and the use of vignettes as a 
teaching and assessment tool in the two courses 
were identical. Course topics included models of 
learning theory, models of teaching theory, the 
application of learning theory to online education, 
and distance learning assessment theory. 

Course assessment included two vignette 
assignments, a midterm, two additional vignette 
assignments, and a final project. The midterm 
included four vignettes to be analyzed as well as 
five essay questions. The final project was the 
creation of a detailed proposal for an online 
course or training program. 


Vignette Assignment Descriptions 

In the first and second vignette assignments 
(abridged), the participants were given descrip- 
tions of teaching situations and asked to classify 
and evaluate one of them in terms of various 
issues of teaching and learning theory. Each vi- 
gnette's description was purposely vague so that 
various interpretations were possible, allowing the 
participant to find clues in the vignette description 
that indicated the presence of a learning strategy. 
In the third and fourth vignette assignments 
(truncated), participants were asked to adapt a 
hypothetical online lesson to fit a specific learning 
theory and then create an assessment plan for 
measuring participant performance. The vignettes 
were scored by the instructor using a scoring 
guide, (see Table A1 in the Appendix), which 
weighed the defense of a choice more heavily than 
a particular choice. These vignette assignments 
were therefore used to measure mastery of course 
content and transfer of those concepts to new 
contexts. 

Results 

To collect evidence of reliability of the vignette 
as an assessment tool, means and correlation 
coefficients were calculated using data for each of 
the participants' four vignette scores, (a cumula- 
tive vignette score with 100 possible points), a 
midterm score (100 possible points), and a final 
project score (100 possible points). An alpha level 
of .05 was used for all statistical tests. 

Mean vignette, midterm, and final project 
scores were not significantly different across the 
two courses. Moreover, when comparing earlier 
sections of each course (1998-2000) to more recent 
sections (2000-2003), mean scores were not signifi- 
cantly different. These findings together suggested 
that the scoring guides for the various assign- 
ments were consistently administered from sec- 
tion to section and from course to course over the 
years and that any differences in the participant 
populations did not significantly affect vignette, 
midterm, or project performances. The data for all 
610 participants were therefore considered collec- 
tively. 

Cumulative vignette scores (M = 91.77, SD = 
6.24) were not significantly different from mid- 
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term scores (M = 92.00, SD = 4.32) or final project 
scores (M = 92.71, SD = 5.22). Intercorrelations 
between cumulative vignette and midterm scores 
were r = .27, p < .0001, between cumulative vi- 
gnette and final project scores r = .21, p < .0001, 
and between midterm and final project scores r = 
.09, p < .03. 

Since there was a significant overlap of course 
content between the vignettes and the midterm, it 
was expected that their pairing would produce the 
greatest correlation coefficient. There was less 
content overlap between the final project and the 
earlier assessments. It was nevertheless notewor- 
thy that the correlation coefficient between the 
vignette scores and the final project scores was 
nearly the same as the vignette-midterm correla- 
tion, while the midterm and final project scores 
produced a much smaller correlation coefficient. 
These findings suggested that the vignette score 
was better than the midterm as a predictor of final 
project performance. 

Vignette scores were then sorted into three 
groups (lower, average, higher) to see if vignette 
scores could successfully predict midterm and 
project scores. Mean midterm and final project 
scores were significantly different among the three 
sorted vignette groups. When midterm scores 
were sorted into three groups using the same 
cutoff values, mean final project scores were not 
significantly different among the three midterm 
groups. These two findings again suggested that 
the vignette score was better than the midterm as 
a predictor of final project performance. 1 

In each section of the two courses, approxi- 
mately 10% of the participants posted incomplete 
responses to the first vignette assignment that 
needed to be resubmitted. These respondents 
initially failed to address all of the tasks in the 
instructions (e.g., not defending their choices with 
vignette clues). In all cases, resubmitted vignette 
scores were substantially higher. For these partici- 
pants, the first vignette assignment evidently 
incorporated scaffolding as part of the learning 
technique. The 10% resubmission figure dropped 
to nearly zero on subsequent vignette assign- 
ments, which suggested that participants quickly 
learned how to complete vignette assignments. 


Discussion 

Preliminary results in two online teacher 
education courses indicated a significant link 
between vignette and midterm scores and sug- 
gested that vignettes may be a reliable assessment 
tool in measuring teacher pedagogical under- 
standing and an area of research worth further 
study. Since 50% of the midterm points were 
vignette items, this correlation was a measure of 
reliability, i.e., the scoring guides were consis- 
tently used and participants tended to perform 
similarly on the two assessments. Since the final 
project was not vignette-related but measured the 
same course content, the correlation between 
vignette and final project scores was a measure of 
construct validity, i.e., evidence that supported 
decisions concerning final projects and overall 
course grades. 

Vignette Assessment as a Learning Event 

The re-submission data provided additional 
support to the notion that the vignette assessment 
is an episode of learning in and of itself. The 
participants quickly learned how to complete 
vignettes and remarked that they understood the 
course content much more clearly following the 
vignette assignment. Wolf (1993) refers to assess- 
ment as a "heads-on encounter with a culture's 
models of prowess. Assessments publish what we 
regard as skill and what we will accept or reject as 
a demonstration of accomplishment" (pp. 213- 
214). An assessment therefore allows students to 
see their work as someone else sees it. 

The familiar assessments, e.g., standardized 
multiple-choice tests, are often the scapegoat 
example of assessment as a missed learning 
opportunity. Students are not always given sam- 
ples of different levels of performance or the 
criteria that define those levels. They rarely see the 
test booklet again but instead are handed a brief 
summary report or, worse, just a score. Few are 
the opportunities to discuss problem-solving 
strategies or attempt a second try. Yet, the re- 
search is clear that worthwhile work requires 
incubation, revision, and collaboration. 

Vignette assignments are particularly well- 
suited to serving two learning purposes: as an 
opportunity to learn, they encourage reflection. 
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rehearsal, motivation, and collaboration; and as an 
assessment tool, the detailed analysis behind the 
score allows for re-submitted work and further 
reflection. Wolf (1993) refers to a zig-zag path 
between assessment and instruction, considering 
which assessment practices will protect, nudge, 
and inform the students during the long course of 
work. As learners plot a course between humility 
and excellence, the assessment becomes an epi- 
sode in which students learn how to write, con- 
duct research, experiment, or solve problems 
along this zig-zag path. 

One benefit of the vignette assignment is that 
the target of the assessment shifts so that instruc- 
tors and learners rate performances. Assessments 
can promote higher-order critical thinking skills 
just as well as lectures or activities. The assess- 
ment episode is not necessarily a terminal one; the 
learning goes on. They need not be individualistic; 
collaboration keeps assessment dynamic. And the 
need for test security and authentic authorship is 
lessened; instructors can capture a much broader 
and more thorough set of snapshots of the learn- 
ers' performances. As a result, the assessment's 
reliability is strengthened, making its inferences 
more valid. 

Suggestions for Further Study 

Based on these initial results we believe the 
vignette to be a highly effective assessment tool 
which has a strong predictive component and 
provides a rich learning experience for eliciting 
and promoting teacher understanding. Herman's 
(1998) case vignette research, involving a related 
instructional tool in a similar setting, suggests that 
vignettes help teachers apply theoretical con- 
structs and research findings to classroom situa- 
tions. We encourage teacher educators and re- 
searchers to engage in further study on the use of 
vignettes in assessing pedagogical reasoning in 
teacher education programs. Trends worthy of 
further investigation include vignette effectiveness 
in formative and summative assessment, in assess- 
ing problem-solving skills, and in assessing trans- 
fer of knowledge and skills from one educational 
setting to another. 

Further research is suggested to isolate the 
study of vignettes from the following variables: 


• delivery mode (face-to-face, online, blend- 
ed, text-based, video) 

• instructional method (lecture, instructor- 
moderated discussion, learner-generated 
summaries) 

• assessment type (learner-completed vi- 
gnettes, learner-generated vignettes) 

• vignette type (truncated vs. abridged) 

• time mode (synchronous vs. asynchron- 
ous) 

• collaboration (cooperative learning vs. 
collaborative assessment) 

• content area (other than instructional 
development) 

Furthermore, the following research issues need to 
be addressed: 

• refining the definition of vignette 

• refining the vignette scoring guide, includ- 
ing equating two or more vignettes used 
in the same assessment 

• collecting evidence of the reliability of 
vignettes as an instructional and assess- 
ment tool 

• collecting evidence of the validity of infer- 
ences made following vignette assessment, 
including course grades and teacher quali- 
fication decisions 

• comparing vignette assessments with 
essays, summaries, and forced-choice test 
items 

Conclusion 

The beauty of the vignette activity is that, by 
its very nature, learners must transfer their learn- 
ing to other situations and in doing so integrate 
their knowledge and skills well enough to make 
predictions about new situations. It is important to 
provide a scoring guide to the participants before- 
hand as well as some sample vignette responses to 
give them a clear indication of what comprises an 
effective vignette analysis. Although this entails 
time spent away from teaching course content, the 
potential for making the assessment activity an 
episode of learning and for encouraging transfer 
of learning to new situations is well worth the 
extra time. 
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Footnotes 

dt was possible that the intercorrelations were 
significant due to the large data sample, (n = 610). 
Thirty random samples of 60 cases (~10% of the 
data sample) were therefore selected and ana- 
lyzed. In 50% of the random samples, cumulative 
vignette scores were significantly correlated with 
final project scores and with midterm scores, ( p < 
.05). Midterm scores however were significantly 
correlated with final project scores in only 17% of 
the random samples. These findings again sug- 
gested that the cumulative vignette score was 
better than the midterm as a predictor of final 
project performance and was an effective predic- 
tor of midterm performance as well. 
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Appendix 

Table A1 

Vignette Scoring Guide 


Language (2 points) 

Language and phrasing are appropriate 
Construction underscores and enhances meaning 
Spelling is correct 
Punctuation is accurate 
Grammar and usage are correct 

Comprehensiveness (3 points) 

Addresses the question 
Answers all parts of the question 
Includes appropriate references 

Addresses at least two points of view when appropriate, including clear and focused statement of 
agreement/ disagreement 

Accuracy (2 points) 

Answers accurately portray the information 
Content is accurate 
Interpretation is accurate 

Defense (3 points) 

Includes relevant evidence in support of each viewpoint 

Justifies answers using appropriate references to readings, theory, and research 
Includes relevant and accurate definitions and components of key terms 
Provides appropriate examples of key terms and issues 
Defines the problem and suggests viable resolutions 


Final Score (10 points) 

9-10 = A 

8 = B 

6-7= C 
5 = D 

0-4= F 
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